|
Grubb’s Econometrics
Grubb’s econometric evidence consists of five tables of results characterizing the time series
behavior of early colonial and state prices and exchange rates in such a way as to suggest that
prices and exchange rates behaved benignly before the mid-1790s, when colonial/state-issued
paper money predominated, and badly thereafter, due to the transition to the dollar.
Our position is that these econometric exercises serve no useful purpose, and do not, and could
not, establish Grubb’s proposition, even had the econometrics been performed flawlessly (which,
in our view, it was not). We hold this opinion for the following reasons.
First, the tables for the most part are devoted to comparing the behavior of macroeconomic time
series before and after the mid-1790s. Whenever Grubb finds “statistically significant” structural
breaks in the mid-1790s, he attributes them to the transition to the dollar. Our view, which is
developed at length in the published version of the paper, is that the only aspect of the “transition
to the dollar” that might plausibly have occurred in the mid-1790s was a simple change in the unit
of account -- the more substantive changes (most notably, the emergence of banks and the
disappearance of colony/state-issued bills of credit from circulation) generally came earlier. That a
mere change in the unit of account could drastically alter the macroeconomic behavior of the
American economy is implausible; if a discontinuity exists in the mid-1790s, it must be due to
other factors.
The second kind of evidence Grubb presents are various unit root tests. For example, for the
period after the transition, Grubb cannot reject a unit root in purchasing power parity
relationships; he takes this finding as evidence of macroeconomic instability due to the new
monetary arrangements. Other unit root tests Grubb performed on “pre-transition” data usually
reject unit roots, which he interprets as evidence of greater stability in the earlier era. There are at
least two difficulties with this line of argument. Despite the fact that many time series practitioners
use the presence or absence of unit roots as a proxy for judging whether economic relationships
are unstable or not, the connection is a very dubious one. McCallum (1993, pp. 34-38) presents a
persuasive case, in our view, that the presence or absence of unit roots in such economic data is
essentially an irrelevant consideration.
The second difficulty is that most of Grubb’s data series are extremely short, frequently less than
two dozen observations. Even in samples of size 50, Dickey-Fuller tests have meager power
against alternatives in the general vicinity of a unit root (Dickey, Bell, and Miller, 1986, p. 18).
Even the size of Dickey-Fuller tests can be wildly inaccurate if the true series contains a moving
average component (Schwert, 1989). Moreover, Grubb -- unavoidably -- had to perform
preliminary tests to choose his lag lengths and to decide whether to include deterministic trends,
and these tests had to be performed using the same scanty data. All Grubb’s statistical findings,
weak as they would be if these factors were known a priori, are subject to further qualification
because their accuracy is contingent upon the correct determination of lag lengths and trends. In
short, we think it impossible to draw meaningful conclusions about unit roots from such short
time series and also consider the entire unit root question to be an irrelevant one, for the reasons
detailed by McCallum (1993).
Given our conviction that Grubb’s econometric findings are essentially irrelevant, we declined –
to the disappointment of our referees – to enter into an a full-fledged econometric duel. However,
we recognize that many readers may have a different view of the methodological issues, and
hesitate to discountenance a paper utilizing modern econometrics in favor of an alternative view
based entirely on narrative history. Therefore, we conclude with a few comments on Grubb’s
econometrics.
Grubb’s Table 3 (2003, p. 1793) is arguably the most important in his paper, for it tests for a
structural break in the time series behavior of prices coincident with the transition to the dollar.
We believe that Grubb’s conclusions are highly sensitive to the precise timing of the structural
break, that the dates he used were quite favorable to his case, and that the choice of dates was
debatable.
According to Grubb’s notes: “For U.S. states, D is a structural break variable representing the
transition from prices quoted in state currency to prices quoted in the U.S. dollar. It is coded as
one for years greater than 1792 in Massachusetts, 1794 for New York and Pennsylvania, and
1795 for Maryland, Virginia, and South Carolina, and zero otherwise.” The dating of the
structural break is of course crucial because placing the break in different years produces very
different econometric results. At one point, Grubb writes (2003, p. 1782) that the behavior of
prices, as revealed in his Figure 1 and Figure 2, can be used to “identify when markets shifted
from state currency to U.S. dollars.”
Had the dates been determined in this manner, there would be a serious issue with pretest bias.
Elsewhere though, Grubb states that he dated the transitions by determining the date at which the
unit of account changed. Only in the case of Pennsylvania, however, does he detail the source
consulted to determine the date, and in Pennsylvania, his own sources suggest the transition ought
to be placed in 1796, not 1794. When the equation is estimated using 1796 and not 1794, the
t-statistic on the structural break dummy falls from 5.59 to 1.29 (although an interaction term
with the trend remains significant).
It is also difficult to see how Grubb settled on 1792 as the date of the transition in Massachusetts.
Winifred B. Rothenberg’s article (1979), his source for the data, says nothing about when the unit
of account changed in Massachusetts, and since Rothenberg’s price index was constructed using
data gleaned from two dozen account books, a diary, and 350 probate inventories, Grubb could
scarcely have relied on her original sources for his dating. The one secondary source cited by
Grubb that does date changes in the unit of account is Cole (1938, p.115), which reports Boston
wheat prices were still being quoted in shillings in 1798. Moreover, in 1795 Massachusetts passed
a law dictating that dollars and cents would be the money of account after 1 September 1795, a
measure that would have been unnecessary had the unit of account changed in 1792
(Massachusetts, 1795, chap. 18, pp. 452-3). Placing the Massachusetts structural break dummy in
1792, as Grubb does, results in a t-statistic of 3.59, but it drops to 2.68 with the dummy in 1793,
0.45 in 1794, -0.29 in 1795, -0.47 in 1796, and -0.81 in 1797.
Cole (1938) does not support Grubb’s transition dates for Pennsylvania, New York, or South
Carolina either. Philadelphia prices were almost all quoted in dollars as early as 1792. New York
prices were quoted in pounds, shillings, and pence until 1796. Charleston prices were quoted in
pounds, shillings and pence until 1803. (Cole has no data on Virginia or Maryland.) Using Cole’s
dating for Pennsylvania, New York and South Carolina reduces Pennsylvania and South
Carolina’s structural breaks to statistical insignificance, leaving only New York (where the date
shifts only a year) as statistically significant. Thus Grubb’s conclusions regarding the statistical
significance of the structural break are sensitive to his choice of break years, and his choices are
debatable.
Grubb’s Table 4 (2003, p. 1794) seeks to establish that the inflation rate was more variable after
the mid-1790s than earlier. To this end, Grubb presents point estimates of standard deviations of
the inflation rate, along with estimated standard errors for each state in each of the following three
periods: 1750-1775, 1785-1792, and 1797-1811. The observed pair-wise differences in standard
deviations across periods are declared to be statistically significant at the 10% level in all cases but
one.
The distribution theory underlying the claim of statistical significance, however, is flawed. The
first, comparatively minor problem, is that the standard errors of the sample standard deviations
are produced using a bootstrap method that assumes inflation rates are statistically independent
over time. Grubb’s Table 1 (2003, p. 1784) is devoted to establishing the time series properties of
the inflation rate, and he never finds prices to be a random walk, implying his bootstrap estimates
of the standard errors are doubtful. This, however, is but a quibble compared to another error in
the same table. If one examines Table 4 carefully, one can see that Grubb reports the standard
deviation of the Massachusetts exchange rate to be 0.0791 in 1785-1792, with a standard error of
.0147, while in 1797-1811, the standard deviation is reported to be 0.0956, with a standard error
of 0.0184. Those point estimates differ by only 0.0162, less than one standard error of the
estimate for 1797-1811 alone. The difference, Grubb says, is statistically significant at the 10%
level, although that is clearly impossible.
Grubb misapplied a t-test designed for testing the difference of population means; consequently,
all the comparisons in Table 4 are seriously biased towards rejection of the null. In an email,
Grubb explained that the calculation was performed in this manner:
An argument might be made for the asymptotic normality of a statistic similar to this one, but one
would have to omit n1 and n2 from the calculation.
If one were willing to assume that the annual inflation rates were statistically independent over
time (an assumption Grubb makes implicitly in his bootstrapping) and that the inflation rates were
draws from independent normally distributed populations (a requirement for the t-test on means
Grubb misapplies) we could fix this serious error and perform a correct test using the fact that the
ratio of the sample variances would have an F distribution under the null hypothesis of equal
population variances. In the comparison of 1797-1811 with 1785-1792, where Grubb finds all the
differences save one (Pennsylvania) statistically significant, the F-test (using alpha = .10, as Grubb
does) finds all the differences save Maryland statistically insignificant. In the comparison of
1797-1811 with 1750-1775, Grubb finds each of the differences to be significant, although two of
the differences (Massachusetts and South Carolina) go in the “wrong direction” in the sense that
variability was greater in colonial than in federal times. Comparing 1797-1811 to 1750-1775 using
the F distribution and alpha = .10, New York and South Carolina become insignificant. Using the
more conventional alpha = .05 would also make the cases of Massachusetts and Maryland
insignificant.
Even these tests are problematic, because the F test is known to be sensitive to departures from
normality, and because Grubb provides considerable evidence suggesting that these annual
inflation rates are serially correlated over time. Some kind of test could probably be constructed
to overcome these difficulties, but doing so would require considerable effort on our part, and
would only serve to persuade the reader that we viewed the matter to be one of great importance,
which we do not. After all, once the lack of independence is recognized, even the appropriate null
hypothesis is unclear – should one test whether the unconditional variances of inflation are equal,
or whether the innovations in the variances of inflation are equal? Therefore we shall not carry the
matter further than to conclude that the results reported in Table 4 are incorrect and that they
greatly exaggerate the evidence suggesting inflation was more variable after 1797 than before.
Grubb’s Table 5 (2003, p. 1795) is intended to demonstrate that after the mid-1790s estimates of
a purchasing power parity relationship between each of the American states and England show
purchasing power parity failing, in the sense that one cannot reject the presence of a unit root. As
a general matter, accepting any null hypothesis does not prove the null hypothesis is true. That
observation is especially apt in this instance; for a sample sizes of 50, the Dickey-Fuller test is
known to have low power against alternatives anywhere near the unit circle, and Grubb’s sample
sizes never exceed 17 (Dickey, Bell, and Miller, 1986, p. 18). All the tests in this table, however,
should be disregarded, because Grubb used an inappropriate exchange rate series. Grubb found
his exchange rate data in Officer (1996), but Officer had processed the data in various ways, most
notably by making sizable adjustments (Officer, 1996, p. 77, table 6.5) to correct the exchange
rate for the inflation in England during the period of the paper pound (1797-1815). Such
adjustments are inappropriate for a test of purchasing power parity, but Grubb failed to note them
and back them out.
We mention this as yet another illustration of the weakness of Grubb’s econometrics, not because
using the correct series results in striking rejections of unit roots. Like McCallum (1993) we
believe the presence of unit roots is irrelevant to the issue at hand, and like most sensible
statisticians, we recognize that accepting a null hypothesis, particularly when using a test
practically devoid of power against plausible alternatives, cannot be construed as proof that the
null hypothesis is true.
|